1. Field of the Invention
The present invention relates to a method and system for accessing a memory device and, more particularly, to a method and system for access arbitration between multiple channels and/or devices desiring access to the memory device.
2. Description of the Related Art
Host bus adapters (HBAs) are well-known peripheral devices that handle data input/output (I/O) operations for host devices and systems (e.g., servers). In simple terms, a HBA provides I/O processing and physical connectivity between a host device and external data storage devices. The storage may be connected using a variety of known “direct attached” or storage networking technologies, including Fibre channel, iSCSI, VI/IP, FICON, or SCSI. HBAs provide critical server CPU off-load, freeing servers to perform application processing. HBAs also provide a critical link between storage area networks (SANs) and the operating system and application software residing within the server. In this role the HBA enables a range of high-availability and storage management capabilities, including load balancing, SAN administration, and storage management.
In recent years, the need for host CPU I/O off-load has become increasingly important because of the wide adoption of storage area networking (sharing storage among multiple hosts within a network), requiring complex and intensive I/O operations, and the emergence of processing intensive I/O protocols such as iSCSI. Historically, I/O data rates increased at approximately the rate of Moore's law, which allowed servers to maintain I/O processing performance from one product generation to the next. Networking technologies, however, have increased data rates in ten-fold increments. The adoption of storage networking has driven increase in storage I/O data rates closer to those of traditional networking technologies, with increases of 4 to 10 times. This trend shows no signs of slowing down as iSCSI and Fibre Channel technologies prepare to support 10 Gb/s data rates, and higher, in the future.
The accelerating I/O data rates is creating an “I/O processing gap” in which the computing power needed to fill a fast “data pipe” is growing faster than the CPU power available in the server. Without sufficient processing power, a high speed storage network is under-utilized. Therefore, an HBA is needed to fill the I/O processing gap and provide the performance and efficiency that storage networking can deliver. Additionally, because HBAs offload the server's CPU from managing I/O operations, valuable CPU cycles are returned to the server for application processing, which an end user is primarily concerned with.
FIG. 1 illustrates a block diagram of a host system 10, which includes a conventional host server 12 that executes application programs 14 in accordance with an operating system program 16. The server 12 also includes necessary driver software 18 for communicating with peripheral devices. The server 12 further includes conventional hardware components 20 such as a CPU (not shown), host memory, e.g., ROM or hard disk drive, (not shown), RAM (not shown), cache (not shown), etc., which are well known in the art.
The server 12 communicates via a peripheral component interconnect (PCI or PCIX) bus interface 22 to a HBA 24, which handles the I/O operations for transmitting and receiving data to and from remote fibre channel disk storage devices 28 via a fibre channel fabric 26. Other host systems 30 may also be “connected” to the fibre channel fabric 26 via respective HBAs 32 in a similar fashion. The server 12 may communicate with other devices 36 and/or clients or users (not shown) via an Ethernet port/interface 38, for example, which can communicate data and information in accordance with well-known Ethernet protocols. Various types of communication ports, interfaces and protocols are also known in the art that may be utilized by the server 12. The server 12 may also be connected to the Internet 40 via communication port/interface 38 so that remote computers can communicate with the server 12 using well-known TCP/IP protocols. Additionally, the server 12 may be connected to local area networks (LANs) (not shown) and/or wide area networks (WANs) (not shown) in accordance with known computer networking techniques and protocols.
FIG. 2 is a block diagram illustrating some core components of a conventional HBA card 24. The HBA 24 includes a plurality of contact pins or slides 25 for making electrical contact to corresponding pins or slides in a PCI or PCIX bus slot (not shown) located on the PCI(x) bus interface 22 (FIG. 1), thereby allowing the server 12 (FIG. 1) to communicate with the HBA 24. The HBA 24 further includes a controller circuit 50 that includes a microprocessor 52 for executing program instructions and processing data stored in volatile memory 54. The HBA card 24 also includes a fibre channel connector/port 54 for establishing a communication link to the fibre channel fabric 26.
In conventional systems, typically, the microprocessor 52 is responsible for operations such as direct memory accesses between the host memory and the HBA. The HBA controls data transfers to and from system memory (e.g., FC Disk 28) without the need for host CPU involvement. Additionally, the microprocessor 52 is responsible for handling other I/O data management and processing operations such as processing context information for each frame of data, routing data to appropriate storage devices 28, etc. These types of functions by the microprocessor 52 and HBA 24 are well-known in the art.
The executable program and data (collectively referred to herein as the “program image”) necessary for the microprocessor 52 to perform its various I/O management and handling operations is stored in a nonvolatile memory 56 (e.g., flash memory 56). This program image, or at least portions thereof, and data stored in the flash 60 are downloaded on an as needed basis, via flash interface device 58 and a flash interface bus 60, to the volatile memory 54 and/or cache (not shown) for faster execution and processing by the microprocessor 52.
During normal operation, in order to access (read or write) the flash memory 56, during a host or HBA wake-up process, for example, the microprocessor 52 writes to and sets dedicated registers 62 (referred to herein as μp registers 62) allocated for this purpose. As explained in further detail below with respect to FIG. 5A, the μp registers 62 typically include a read-address register 62a (FIG. 5A), a read-data register 62b, a read-next-data register 62c, a write-address register 62d, a write-data register 62e, and a control/status register 62f. Techniques for accessing flash memory utilizing these types of registers and conventional control signals are well-known in the art.
FIG. 3 illustrates a block diagram of some of the core components or modules of a program image that are downloaded from flash memory 56 to volatile memory 54. As shown in FIG. 3, the program image includes a kernel program or module 70, which is the core or minimum instruction set for the microprocessor 52 to operate and execute other instructions and functions. Such kernel programs are well-known in the art. A basic input/output system (BIOS) program 72 is also stored within the volatile memory 54 for performing HBA component configuration and initialization operations at system start up or wake up. Such BIOS programs are also well-known in the art. The program image further includes a status and error program 74 for monitoring the status of microprocessor 52 operations and reporting an error signal if an error occurs. Such status and error programs are also well-known in the art. The program image also includes one or more executable programs 76 and data 78 for performing its I/O operations as discussed above. The diagram in FIG. 3 illustrates that different areas of the flash hold different kinds of contents. Generally, there are two types of accesses to flash memory—automatic and programmed. Automatic accesses are carried out because the requesting device needs the information to be functional. In particular, the microprocessor must read instructions from the flash device directly into its instruction-processing pipeline (not copied to any other memory) when it starts processing after reset is cleared. The “kernel” area contains these instructions. This design relies on information contained in the non-volatile memory to provide a wake-up program for the microprocessor. In a multi-channel HBA, the microprocessor in one channel may be reset and restarted independent of the other microprocessor(s).
The “BIOS” area has another type of automatic access. The host system (server) reads from the flash across the PCI/X bus a program that extends its built-in wake-up program. The program is tailored to the HBA to allow the host system to use the HBA I/O facilities to “boot up,” that is, to read extensive control programs from a large external data source. Without this non-volatile “Expansion ROM” the HBA is useless to the host system until the entire host system OS becomes active by booting up over a different external access facility. These automatic direct host-to-flash accesses are known in the art as host ROM base address register (BAR), or ROM BAR, accesses.
The second type of access is a programmed access. In contrast to “automatic” access, described above, programmed accesses retrieve data for various uses in accordance with various application programs. Additionally, these programmed accesses can wait (allowing the requesting device to do other things in the interim) on data availability. In one embodiment, register sets having read data valid (RDV) bits, described in further detail below, are implemented to indicate that data as valid and available in accordance with this programmed access protocol.
Regardless of the way reads are carried out, however, any attempt to alter the non-volatile contents of the flash memory requires a series of programmed accesses first to start an operation (erase sector or write word) and then to determine when the operation is complete. Because the flash device changes from a memory device to an operational controller during these operations, any read access by a requester other than the program that initiated the operation would yield incorrect data.
In order to further meet the increasing demands of I/O processing applications, multi-processor architectures have been developed to provide multi-channel and/or parallel processing capability, thereby increasing the processing power and speed of HBAs. In prior multi-processor HBAs, the function of arbitrating which microprocessor 52 had access to flash memory 56 at any given time was performed by hardware within the controller 50. In contrast to a flexible software control, hardware control is typically implemented by a set of logic gates, which are fixed in their behavior. These gates have no register control bits for feedback and control of the behavior of respective circuits and/or data paths. Thus, in prior multi-processor HBAs, each microprocessor 52 generally had sole access to a separate flash memory 56. If the flash memory 56 was accessed by more than one microprocessor 52, hardware within the controller 50 generally was switched to service a single requester. In other words, hardware within controller circuitry 50 was able to support only one requester at a time, requiring an external control to switch it to service a different requester. These fixed hardware control methods did not allow for flexibility or modification of arbitration schemes once they had been implemented in a HBA device. For example, in prior systems, only one processor had access to the flash memory 56 at any given time in accordance with a fixed access scheme (e.g., round robin, first-in-first-out). Additionally, a source (e.g., host device application) desiring access to the flash had to communicate with a respective microprocessor to access respective areas of the flash 56 that were mapped for access to that particular processor, for example, when each processor had a separate dedicated flash device. Therefore, prior art methods did not provide flexibility or maximize efficiency in sharing flash memory 56.
In order to provide further HBA functionality, robustness and flexibility, some HBAs provide direct access to the flash memory 56 to a host device through a communication path that is independent of the one or more microprocessors 52 on the HBA, and separate from automatic host ROM BAR access requests during host system start up, as described above. For example, if the one or more microprocessors 52 are not functioning properly due to a corrupt “program image,” it is desirable to allow a host device to directly access the flash 56 and reprogram a new program image for downloading to the volatile memory 52, from where it is executed by a respective microprocessor 52. A method and system for providing programmed direct host-to-peripheral memory access is described in co-pending application Ser. No. 10/651,887, entitled “Direct Memory Access From Host Without Processor Intervention.”
Additionally, it is also possible to have multiple parallel direct host-to-flash communication paths for providing direct access to the flash 56 without microprocessor 52 intervention. Thus, there are at least three categories of requesters that may desire access to the flash 56. The first category includes one or more HBA microprocessors. The second category includes automatic host ROM BAR accesses during host system start up, and the third category includes direct host-to-flash programmed accesses to flash. In an HBA having multiple microprocessors 52 and multiple host communication paths or channels desiring access to the flash memory 56, it is necessary to provide a protocol for arbitrating access requests to flash by these multiple requesters.
As mentioned above, prior art arbitration methods based on hardware control do not provide flexibility and the ability to modify and customize arbitration schemes in accordance with an individual system's design and use. Thus, there is a need for a method and system that provides a flexible arbitration scheme capable of handling and arbitrating multiple requests to access shared memory. There is a further need for an arbitration method and system capable of being modified and customized in accordance with a desired protocol or design for individual systems.